Goto

Collaborating Authors

 Manisa




UM_FHS at TREC 2024 PLABA: Exploration of Fine-tuning and AI agent approach for plain language adaptations of biomedical text

arXiv.org Artificial Intelligence

This paper describes our submissions to the TREC 2024 PLABA track with the aim to simplify biomedical abstracts for a K8 - level audience (13 - 14 years old students). We tested three approaches using OpenAI's gpt - 4o and gpt - 4o - mini models: baseline prompt engineering, a two - AI agent approach, and fine - tuning. Adaptations were evaluated using qualitative metrics ( 5 - point Likert scales for simplicity, accuracy, completeness, and brevity) and quantitative readability scores (Flesch - Kincaid grade level, SMOG Index). Results indicate d that the two - agent approach and baseline prompt engineering with gpt - 4o - mini models show superior qualitative performance, while fine - tuned models excelled in accuracy and completeness but were less simple. The evaluation results demonstrated that prompt engineering with gpt - 4o - mini outperforms iterative improvement strategies via two - agent approach as well as fine - tuning with gpt - 4o. We intend to expand our investigation of the results and explore advanced evaluations.


Association rule mining with earthquake data collected from Turkiye region

arXiv.org Artificial Intelligence

Earthquakes are evaluated among the most destructive disasters for human beings, as also experienced for Turkiye region. Data science has the property of discovering hidden patterns in case a sufficient volume of data is supplied. Time dependency of events, specifically being defined by co-occurrence in a specific time window, may be handled as an associate rule mining task such as a market-basket analysis application. In this regard, we assumed each day's seismic activity as a single basket of events, leading to discovering the association patterns between these events. Consequently, this study presents the most prominent association rules for the earthquakes recorded in Turkiye region in the last 5 years, each year presented separately. Results indicate statistical inference with events recorded from regions of various distances, which could be further verified with geologic evidence from the field. As a result, we believe that the current study may form a statistical basis for the future works with the aid of machine learning algorithm performed for associate rule mining.


Are Deep Learning Classification Results Obtained on CT Scans Fair and Interpretable?

arXiv.org Artificial Intelligence

Following the great success of various deep learning methods in image and object classification, the biomedical image processing society is also overwhelmed with their applications to various automatic diagnosis cases. Unfortunately, most of the deep learning-based classification attempts in the literature solely focus on the aim of extreme accuracy scores, without considering interpretability, or patient-wise separation of training and test data. For example, most lung nodule classification papers using deep learning randomly shuffle data and split it into training, validation, and test sets, causing certain images from the CT scan of a person to be in the training set, while other images of the exact same person to be in the validation or testing image sets. This can result in reporting misleading accuracy rates and the learning of irrelevant features, ultimately reducing the real-life usability of these models. When the deep neural networks trained on the traditional, unfair data shuffling method are challenged with new patient images, it is observed that the trained models perform poorly. In contrast, deep neural networks trained with strict patient-level separation maintain their accuracy rates even when new patient images are tested. Heat-map visualizations of the activations of the deep neural networks trained with strict patient-level separation indicate a higher degree of focus on the relevant nodules. We argue that the research question posed in the title has a positive answer only if the deep neural networks are trained with images of patients that are strictly isolated from the validation and testing patient sets.


DA-LSTM: A Dynamic Drift-Adaptive Learning Framework for Interval Load Forecasting with LSTM Networks

arXiv.org Artificial Intelligence

Load forecasting is a crucial topic in energy management systems (EMS) due to its vital role in optimizing energy scheduling and enabling more flexible and intelligent power grid systems. As a result, these systems allow power utility companies to respond promptly to demands in the electricity market. Deep learning (DL) models have been commonly employed in load forecasting problems supported by adaptation mechanisms to cope with the changing pattern of consumption by customers, known as concept drift. A drift magnitude threshold should be defined to design change detection methods to identify drifts. While the drift magnitude in load forecasting problems can vary significantly over time, existing literature often assumes a fixed drift magnitude threshold, which should be dynamically adjusted rather than fixed during system evolution. To address this gap, in this paper, we propose a dynamic drift-adaptive Long Short-Term Memory (DA-LSTM) framework that can improve the performance of load forecasting models without requiring a drift threshold setting. We integrate several strategies into the framework based on active and passive adaptation approaches. To evaluate DA-LSTM in real-life settings, we thoroughly analyze the proposed framework and deploy it in a real-world problem through a cloud-based environment. Efficiency is evaluated in terms of the prediction performance of each approach and computational cost. The experiments show performance improvements on multiple evaluation metrics achieved by our framework compared to baseline methods from the literature. Finally, we present a trade-off analysis between prediction performance and computational costs.


Optimization of Residential Demand Response Program Cost with Consideration for Occupants Thermal Comfort and Privacy

arXiv.org Artificial Intelligence

Residential consumers can use the demand response program (DRP) if they can utilize the home energy management system (HEMS), which reduces consumer costs by automatically adjusting air conditioning (AC) setpoints and shifting some appliances to off-peak hours. If HEMS knows occupancy status, consumers can gain more economic benefits and thermal comfort. However, for the building occupancy status, direct sensing is costly, inaccurate, and intrusive for residents. So, forecasting algorithms could serve as an effective alternative. The goal of this study is to present a non-intrusive, accurate, and cost-effective approach, to develop a multi-objective simulation model for the application of DRPs in a smart residential house, where (a) electrical load demand reduction, (b) adjustment in thermal comfort (AC) temperature setpoints, and (c) , worst cases scenario approach is very conservative. Because that is unlikely all uncertain parameters take their worst values at all times. So, the flexible robust counterpart optimization along with uncertainty budgets is developed to consider uncertainty realistically. Simulated results indicate that considering uncertainty increases the costs by 36 percent and decreases the AC temperature setpoints. Besides, using DRPs reduces demand by shifting some appliance operations to off-peak hours and lowers costs by 13.2 percent.


Stress Test for BERT and Deep Models: Predicting Words from Italian Poetry

arXiv.org Artificial Intelligence

In this paper we present a set of experiments carried out with BERT on a number of Italian sentences taken from poetry domain. The experiments are organized on the hypothesis of a very high level of difficulty in predictability at the three levels of linguistic complexity that we intend to monitor: lexical, syntactic and semantic level. To test this hypothesis we ran the Italian version of BERT with 80 sentences - for a total of 900 tokens - mostly extracted from Italian poetry of the first half of last century. We used then sentences from the newswire domain containing similar syntactic structures. The results show that the DL model is highly sensitive to presence of non-canonical structures. However, DLs are also very sensitive to word frequency and to local non-literal meaning compositional effect. This is also apparent by the preference for predicting function vs content words, collocates vs infrequent word phrases. In the paper, we focused our attention on the use of subword units done by BERT for out of vocabulary words. NTRODUCTION In this paper we report results of an extremely complex task for BERT: predicting the masked word in sentences extracted from Italian poetry of beginning of last century, using the output of the first projection layer of a Deep Learning model, the raw word embeddings. We decided to work on Italian to highlight its difference from English in an extended number of relevant linguistic properties. The underlying hypothesis aims at proving the ability of BERT [1] to predict masked words with increasing complex contexts. To verify this hypothesis we selected sentences that exhibit two important features of Italian texts, non-canonicity and presence of words with very low or rare frequency. To better evaluate the impact of these two factors on word predictability we created a word predictability measure which is based on a combination of scoring functions for context and word frequency of (co-)occurrence. The experiment uses BERT assuming that DNNs can be regarded capable of modeling the behaviour of the human brain in predicting a next word given a sentence and text corpus - but see the following section. It is usually the case that paradigmatic and syntagmatic properties of words in a sentence are tested separately.


Semantic Web Enabled Geographic Question Answering Framework: GeoTR

arXiv.org Artificial Intelligence

With the considerable growth of linked data, researchers have focused on how to increase the availability of semantic web technologies to provide practical usages for real life systems. Question answering systems are an example of real-life systems that communicate directly with end users, understand user intention and generate answers. End users do not care about the structural query language or the vocabulary of the knowledge base where the point of a problem arises. In this study, a question answering framework that converts Turkish natural language input into SPARQL queries in the geographical domain is proposed. Additionally, a novel Turkish ontology, which covers a 10th grade geography lesson named Spatial Synthesis Turkey, has been developed to be used as a linked data provider. Moreover, a gap in the literature on Turkish question answering systems, which utilizes linked data in the geographical domain, is addressed. A hybrid system architecture that combines natural language processing techniques with linked data technologies to generate answers is also proposed. Further related research areas are suggested.


Online Decentralized Frank-Wolfe: From theoretical bound to applications in smart-building

arXiv.org Artificial Intelligence

The popularity of sensors and IoT devices has the potential of generating and equivalently accumulating data in order of Zeta bytes [1] annually. High throughput, low latency, data consumption, networking dependencies are often the key metrics in designing high-performance learning algorithms under the constraint of low powered computing. In recent times, there has been an alternate trend to process data on cloud or dump into a centralized database. Commonly known as edge computing, the new paradigm embraces the idea of using interconnected computing nodes to reduce high bandwidth consuming data uploads, privacy preservation of data and knowledge on the fly. Smart building applications typically have a profound implication on environment in terms of energy savings, reduction of green house emission, etc. Predicting the future often forms the basis of corrective actions taken by such apps and can be regarded as a predominant use-case of machine learning.